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ABSTRACT 

The arrival of new media technology has the potential to radically change education. It has become important for 
both academics and practitioners to understand the behavioural intentions of students towards media technology. 
Many scales have been developed to measure the attitudes of secondary students towards the usage of media 
technology. However, few scales have been developed to assess media literacy in primary school students. To 
develop a validated scale for the media literacy of elementary school students, the Media Literacy Self- 
assessment Scale (MLSS) was adopted and modified in this study and then validated using a sample (N=594) of 
students from Taiwan. The MLSS is a two-factor scale that measures learning with media (LWM) and media 
communication & ethics (MCE); these components have been proposed to constitute the multidimensional 
constructs. A confirmatory factor analysis was performed on a proposed nine-item model of the MLSS, and it 
was found the model provides a good fit. Gender and grade comparisons of the results are discussed. Educational 
implications and suggestions for future research are also provided. 

Keywords: Media Literacy Self-assessment Scale (MLSS), Scale Validation, Confirmatory Factor Analysis, 
Taiwanese Elementary School Students 

INTRODUCTION 

The influence of media technology on the social environment is ubiquitous. In today's world, media has become 
a way of life. The media informs us, entertains us, and connects us to the world. Recently, new technology has 
emerged (such as edutainment technology) and been applied in different fields (Chang, Lee, Wang, & Chen, 
2010; Cheng, Wu, Liao, & Chan, 2009; Chiang, Lin, Cheng, & Liu, 2011; Lee & Chen, 2009; Lin & Liu, 2009; 
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Liu, in press; Liu, Cheng, Lin, Chang, & Chen, 2008; Liu & Lin, 2009; Liu, Kou, Lin, Cheng, & Chen, 2008; 
Liu, Lin, & Chang, 2010; Sung, Chang, & Lee, 2008). Through technology, the media is enmeshed in our daily 
lives. The media has the potential to shape personalities and change the way we perceive and understand the 
world and our immediate reality. Moreover, we have seen that a large number of people depend on the Internet to 
collect information, read news, listen to music, download films, play games, and complete work. In recent years, 
the media has become one of the most important channels for the acquisition of knowledge for children in the 
modern world (Bennett, Maton, & Kervin, 2008; Cabra-Torres & Marciales-Vivas, 2009; Liu & Chang, 2010; 
Prensky, 2001; Tapscott, 2009). Therefore, it has become important to learn how to use this powerful technology 
appropriately and to understand what factors affect the individual use of media technology. In the past, 
researchers have found that self-efficacy affects an individual’s use of new technology (i§man & Qelikli, 2009; 
Teo, 2009; Topkaya, 2010). 

Media surrounds us and is present in everything. This phenomenon has made learning easier and more 
interesting for children. Television, radio, computers and the Internet are gradually entering classrooms and 
changing the way that students learn. In particular, computers and the Internet are quickly becoming our 
dominant cultural tools for searching, selecting, gathering, storing, and conveying knowledge in representational 
forms (Covington, 2004; Jenkins, 2006; Kuiper, Volman, & Terwel, 2009). There are both advantages and 
disadvantages to increasing one's knowledge of the different aspects of media. As we adopt the good components 
of this knowledge, we should also try to avoid the bad. The negative messages disseminating through various 
media technologies can be avoided by developing the skills to question, evaluate and analyse these messages. 
Therefore, it is vital for individuals to develop media literacy so that they can make the best use of the new 
technology and so that they are able to interpret and process all kinds of media messages (American Library 
Association, 1989; Enochsson, 2005; Thoman, 2003). 

Media literacy can be broadly defined as a combination of the various skills needed to search, select, analyse, 
evaluate, and communicate in the various forms of media (Considine, Horton, & Moorman, 2009; Enochsson, 
2005; Livingston, 2004; Kuiper, Volman, & Terwel, 2009). Kuiper, Volman and Terwel (2009) identified the 
principal components of three forms of media literacy related to the World Wide Web: Web searching skills, Web 
reading skills, and Web evaluation skills. Similarly, Covington (2004) advocates the notion that media literacy 
involves critical viewing skills and the ability to regard, evaluate, and interpret content. Furthermore, Schaefer 
(2005) pointed out that media literacy is usually conceptualised as a set of skills related to the production of a 
media message. In sum, media literacy has been identified as an essential form of literacy by the Partnership for 
21 st Century Skills, and it is crucial that schools focus on helping students acquire the skills necessary to 
navigate, evaluate, and communicate in various kinds of media (Jenkins, 2006). 

Media literacy does not simply encompass being entertained by the media; it also necessitates that one learns 
something from it. Furthermore, the subjects that we learn in our curricula are also found in the media. These 
subjects may include the arts, science, maths, different languages, social sciences and health. Media literacy may 
also help to develop critical thinking skills. The students are able to get a broad exposure to popular cultural 
references. They can gather statistics and data from the news that can then be the basis for math and science 
learning. Because a major part of the learning process is concentrated on children, the new media can play a 
significant role in satisfying their intellectual curiosity. Children often learn important things through the media, 
which can change their perspective on life. Teachers can also incorporate media analysis whenever the Internet, 
computers, television or video is used in the classroom. 

Scholars have begun to conduct research that focuses on learning in media literacy education and, in particular, 
on the relationships between students’ existing knowledge about the media and the knowledge teachers make 
available. Brag (2002) used her classroom observation to illustrate that what students learn and how they learn it 
during media literacy practice often has little relevance to their everyday media use. Caronia (2009) used 
conversation analysis to identify a typology of the interactions between children while they watched television in 
an educational context. These scholars stress the need to gather ethnographic data on the actual media 
experiences of students and how they perceive media literacy strategies. Erstad and Gilje (2008) explored the 
impact of everyday experiences with media and digital tools on the production practices of students in media 
education. Their survey data indicate that young people largely draw on their media experiences from outside 
school. Therefore, media literacy education should address the intersection between formal and informal ways of 
learning among youth. 

Although media has been around for a long time, many administrators and teachers have just begun to hear about 
media literacy and to realise its importance. Because media literacy education is essential for modern citizens 
and is an important quality in civic society, it should begin at the earliest possible stage (i.e., during elementary 
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school). To equip students with the required skills, three general guidelines exist for teaching basic media 
literacy in K-12 (Utah State Office of Education, 2006) which are (1) Awareness: students will be aware that 
media literacy as a life skill is integral to modem citizenship, informed decision making, and healthy lifestyles. 
(2) Analysis: students will analyze elements of media messages to understand their forms and functions, content, 
and effects on the receiver. (3) Evaluation: students will evaluate elements and intended results of media 
messages to facilitate selection for personal and educational use. Chang and Liu (in press) also proposed three 
components of media literacy for elementary school students: (1) media application skills (students’ abilities to 
perform media technologies), attitudes toward media (students' perceptions regarding ethics of technologies), 
and (3) learning with media (students’ abilities to extract messages from media to perform learning tasks. Based 
on the review of the existing literature, there is no widely accepted instrument to assess media literacy in 
Taiwanese elementary school students. Therefore, the purpose of this study is to develop and validate a Media 
Literacy Self-assessment Scale (MLSS) for assessing the media literacy of elementary school students. 
According to Chang and Liu’s (in press) study, media application skills and attitudes toward media are 
categorized as a combined dimension “media communication & ethics”. Therefore, learning with media (LWM) 
and media communication & ethics (MCE) will be included in the developed instrument to measure students’ 
perceptions towards media literacy. Furthermore, after validating the questionnaire, grade and gender differences 
were analysed. 

DEVELOPMENT AND VALIDATION OF MLSS 
Pilot Study 

Participants 

A total of 300 subjects participated in the pilot study. Subjects were recruited from five elementary schools in 
Taipei, Taiwan. Among the participants, 149 were female students and 151 were male students. There were 146 
5 th graders and 154 6 th graders among the group. 

Reliability and Validity of the MLSS 

A pool of items was adapted from the Media Literacy Self-assessment Scale (MLSS), which was developed by 
Chang and Liu (in press). A total of 13 items were selected to assess each subject's cognitive response towards 
using media technology in learning and their behavioural attitude towards using media technology. All items 
were scored on a 5-point Likert scale that ranged from “strongly disagree” to “strongly agree”. 

The revised 13-item MLSS (See Appendix A) was given in a pilot study to 300 students who were aged 12-13. 
The sample was 49.7% female (n=149) and was drawn from 10 classes in 2 primary schools in Taipei, Taiwan. 
Each survey was administrated by the same person; the MLSS can be administered in 10 min. Participants were 
advised that the scale is not a test and that there are no right or wrong answers. They were asked to indicate the 
level of their agreement with each statement and to answer as honestly as possible. Data was analysed using 
SPSS 12.0 to assess the reliability and validity of the measure. 

To explore the factorial structure of MLSS, we conducted exploratory factor analysis (EFA) on sample 1 
(N=300) using principal axis factoring and varimax rotation (Table 1). Most researchers recommend a sample 
size of 100-200 cases for exploratory factor analysis (Hair, Black, Babin, Anderson, & Tatham, 2006). As such, 
the sample of 300 cases is more than adequate at this stage of analysis. 

The overall alpha coefficient for the 13 items of MLSS was 0.9, and the coefficients for media application skills 
and the media communication and ethics subscales were 0.83 and 0.72, respectively. These coefficients are 
regarded as being acceptable for scale construction (DeVellis, 2003). To choose the number of factors, 
eigenvalues greater than 1 and the screen test were used as the decision criteria. There were two factors that had 
eigenvalues greater than 1; these factors accounted for 49.34% and 8.27% of the variance. An item was retained 
only when it loaded greater than 0.50 on the relevant factor and less than 0.50 on the non-relevant factors. All of 
the factor loadings in each factor were greater than 0.50. The first factor consisted of 6 items (m_l_b, m_2_a, 
m_2_b. m_l_a, m_2_d, and m_2_c with factor loadings of 0.744, 0.693, 0.679, 0.666, 0.576, and 0.513, 
respectively) out of the 13 MLSS items. This factor was named learning with media (the variance of this factor 
explained 49.34% of the total variance). The second factor consisted of 7 items (m_4_b, m_4_a, m_4_c, m_3_a, 
m_3_b. m_l_c, and m_3_c with factor loadings of 0.763, 0.599, 0.576, 0.544, 0.533, 0.528, and 0.522, 
respectively) of the MLSS; this factor was named media communication and ethics (the variance explained 
8.27% of the total variance). 
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Table 1 Results of Exploratory Factor Analysis 


Items 

LWM 

MCE 

m_l_b 

.744 

.286 

m_2_a 

.693 

.363 

m_2_b 

.679 

.272 

m_l_a 

.666 

.328 

m_2_d 

.576 

.300 

m_2_c 

.513 

.453 

m_4_b 

.228 

.763 

m_4_a 

.387 

.599 

m_4_c 

.204 

.576 

m_3_a 

.442 

.544 

m_3_b 

.423 

.533 

m_l_c 

.418 

.528 

m 3 c 

.329 

.522 

Cronbach’s a 

.83 

.72 

Eigenvalues 

6.41 

1.07 

Cumulative of variance (%) 

49.34% 

57.61% 


Note: LWM: learning with media; MCE: media communication and ethics 


Primary Study 

Student Sample 

The participants in this study were elementary school students in Taiwan (5 lh and 6 th graders), whose ages ranged 
from 12 to 13 years old. Over a 5-week data collection period, 900 paper-based questionnaires were distributed 
among three major demographic areas in northern, central, and southern Taiwan. From each area, a varied 
number of individual classes from several elementary schools were chosen to complete a survey. 


The non-responses, unintentional skips or unidentifiable marks on some items of the survey were processed by 
the study as “missing data”. Hence, the valid number of samples for each item or subscale of the survey varies. 
However, the “missing data” does not exceed 5% of the whole data set for any one item or subscale. A total of 
594 usable questionnaires were returned, yielding an effective rate of 66%. Among the respondents, 301 were 
males and 293 were females; 259 were 5 lh graders and 335 were 6 th graders. 

Confirmatory Factor Analysis (CFA) 


Confirmatory factor analysis was conducted to test a two-factor model of the MLSS, as revealed in the pilot 
study. It was hypothesised that confirmatory factor analysis of the MLSS would indicate that the responses to the 
MLSS could be explained by two factors, identified as Learning With Media (LWM) and Media Communication 
& Ethics (MCE). We used Lisrel8.8 with a maximum likelihood method to test our factorial validity. 


We used sample 2 (N=300, randomly selected from the 594 participants) to develop a model. A variety of fit 
indices were used to test the model fit. An adequate model fit is represented by GFI, CFI, and NNFI values that 
are greater than 0.90 (Hoyle & Panter, 1995) and a RMSEA value below 0.05 (Byrne, 2001). Based on the EFA, 
a two-factor model result did not offer a good fit ()f=307.57; GFT=0.86; CFT=0.95; NNFI=0.94; SRMR=0.058; 
RMSEA=0.11). Next, we removed some items to improve the model fit according to the modification index. The 
modification index was computed using residual values. On the basis of this criterion, we deleted items from the 
larger MI in the CFA model. Items m-la, m-2d, m-3b, and m-4b were removed and the fit indices (which shifted 
to x 2 =54.34; GFI=0.96; CFI=0.99; NNFI=0.98; SRMR=0.037; RMSEA=0.06) fit the data well. 

In this study, we employed sample 3 (N=294, selected from the 594 participants after removing the 300 
participants used in sample 2) to further demonstrate a two-factor CFA model. After deleting 4 items of the 
MLSS in sample 2, a two-factor CFA model was also reliable. The results showed a satisfactory fit to the data 
Of=63.52; GFI=0.95; CFI=0.98; NNFI=0.97; SRMR=0.038; RMSEA=0.07). These results indicated that our 
two-factor model had a good fit to the other sample set. 

We next used a competitive model approach to determine the model that best fit the theory. Following that, the 
convergent validity, discriminate validity, composite reliability and item reliability will be shown below. 

Model Competition 


In this stage, in addition to our hypothesis model (first-order, two-factor oblique model), we used an independent 
model and a first-order, two-factor orthogonal model as alternative models. Table 2 presents the independent 
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model that had the worst fit indices. Compared to the independent model, the first-order, two-factor oblique 
model and the first-order, two-factor orthogonal models are significant improvements. Specifically, the first- 
order, two-factor oblique model exhibits the best fit to the data. Because the high-order models did not converge, 
this result indicates that the first-order, two-factor oblique model fits the data best. 


Table 2 The Results of Model Comparisons 


Fit indices 

% 2 (df) 

X 2 /df 

RMSEA 

GFI 

AGFI 

CFI 

NNFI 

SRMR 

Independent model 

1974.02(36) 

54.83 







First-order, two-factor 
orthogonal model 

198.65(27) 

7.35 

.15 

.87 

78 

88 

85 

.27 

First-order, two-factor 

63.52(26) 

2.44 

.070 

.95 

.92 

.98 

.97 

.038 


oblique model 


In a second stage, we further compared the independent model to the one-factor model and the two-factor model 
(Table 3). Specifically, the one-factor model allowed all of the items of the MLSS to load on one factor. The two- 
factor model with the items assigned to the two corresponding variables. Results showed that the two-factor 
model yielded a better fit ( 2 =129.99, p<0.001, df=98, 2 /df=1.32, RMSEA=0.047, GFI=0.90, CFT=0.95, 

NNFI=0.93, SRMR= 0.07), and the A% 2 was also significant. 

Table 3 Results of Different Factor Structure of Models 


Fit indices_y 2 (df) y 2 /df Ay 2 RMSEA GFI AGFI CFI NNFI SRMR 


independent 

model 

1974.02(36) 

54.83 








one-factor 

model 

82.14(27) 

3.04 

1891.88* 

.083 

.94 

.90 

.97 

.96 

.046 

two-factor 

model 

63.52(26) 

2.44 

18.62* 

.070 

.95 

.92 

.98 

.97 

.038 


In conclusion, through the two-stage competitive model, we found that the two-factor oblique model had a better 
fit to the theory according to the dimension of learning with media (LWM) and media communication and ethics 
(MCE). 

Convergent Validity 

The results of our two-factor CFA model are presented in Table 4. All of the factor loadings from the items to 
their latent factors were significant, and the composite reliabilities were all above 0.60. These results provide 
evidence for the convergent validity of our scale. 


Table 4 Results of Convergent Validity of MLSS 


Factors 

items 


Standard 

solution 

R 2 

S.E. 

C.R. 

Composite 

reliability 

AVE 


m_l_b 








Learning 

I understand how to operate media 

.62 

.68 

.46 

.05 

12.35 



with media 

technology devices. 








(LWM) 

m_l_c 









I can select appropriate types of media 
based on my learning needs. 

.67 

.60 

.36 

.06 

10.67 




m_2_a 









I am familiar with the operational functions 
of media equipment that is used to 
broadcast learning content. 

.74 

.78 

.61 

.05 

15.06 

.83 

.50 


m_2_b 









I use different media technology to 
store/backup learning content. 

.76 

.78 

.60 

.05 

14.91 




m_2_c 

I use media for my learning tasks. 

.71 

.67 

.44 

.06 

12.11 




m_3_a 









I understand the content that is conveyed 
by media. 

.64 

.66 

.44 

.06 

11.64 

.75 

.42 

Media 

m_3_c 

.68 

.64 

.41 

.06 

11.17 
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communicat I discuss the displayed contents of media 


ion and 

ethics 

(MCE) 

with others. 
m_4_a 

I possess an accurate understanding of 

.61 

.67 

.45 

.05 

11.86 


media use. 
m_4_c 

I comply with the intellectual property 

.59 

.63 

.39 

.05 

10.85 


rights of media use. 


We followed the procedures proposed by Anderson and Gerbing (1988). Discriminant validity was established 
using chi-square difference tests to compare an unconstrained measurement model with a constrained model (in 
which the correlations between two latent factors are set equal to one). Table 5 indicates that the A%2 was 
significant and that two factors can be discriminated in our scale. 


Table 5 Result of Discriminant Validity 


Variable 

model 


DF 

V/2 


unconstrained model 

63.52 

26 


MLSS 






constrained model 

198.65 

27 

135.13*** 


Student Scores on the Scale 

Table 6 indicates the average item scores and the standard deviations on the two subscales of the MLSS. 
Students scored highest on the media communication & ethics subscale (with an average score of 4.23 per item), 
followed by the learning with media subscale (with an average score of 4.21 per item). The standard deviations 
of both two subscales are moderate and the students’ scores are close to each other. The results indicate that, on 
average, the students demonstrated essential media literacy in a technology-enriched learning environment. 


Table 6 Students’ Score on the MLSS Subscales (N=300) 


Subscale 

Items 

Mean 

SD 

LWM 

5 

4.21 

0.75 

MCE 

4 

4.23 

0.70 


Gender Differences on the Scale 

This study further compares the scores of the male and female students on the two subscales of the MLSS. The 
results of independent t tests are presented in Table 7, revealing that these gender scores on both subscales were 
significantly different at the 0.05 level. Female students expressed more positive perceptions of learning with 
media and of media communications & ethics. In other words, the male students perceived they were less media 
literate. 


Table 7 Gender Comparisons on the Subscales of the MLSS 


Subscale 

Gender 

Mean 

SD 

t 

LWM 

female 

4.38 

.60 

2.66** 


male 

4.17 

.72 


MCE 

female 

4.30 

.67 

2.54* 


male 

4.08 

.85 



Note: N= 149 for female; N=151 for male; *p<0.05, **p< 0.01 


Grade Comparisons on the Scale 

To examine the possible effects of grade level, this study also compared the score of the fifth and sixth graders 
on the two subscales of the MLSS. In general, the latter group scored higher on the two subscales (LWM and 
MCE) than did the former. However, the results of independent t tests revealed that both fifth and sixth graders 
perceived similar levels of competency on each subscale (Table 8). Overall, comparisons of the MLSS scores 
indicated that students at the advanced grade level did not necessarily rate themselves higher in media literacy. 
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Table 8 Grade Comparisons on Subscales of the MLSS 


Subscale 

Grader 

Mean 

SD 

t 

LWM 

5 th 

4.25 

.70 

-.50 


6 th 

4.29 

.64 


MCE 

5 th 

4.15 

.76 

-.88 


6 lh 

4.23 

.78 



Note: N = 146 for the 5 lh graders; N =154 for the 6 th graders; *p < 0.05 


DISCUSSION AND CONCLUSIONS 

To encourage high-quality research, to enable integration and consistency across research studies, and to increase 
our understanding of media literacy, there is a need for a valid and reliable measure of the media literacy 
construct. This study completes an exploratory-confirmatory research cycle by more rigorously validating the 
MLSS. Completing this cycle is important because it required an added precision in the model specification. 
From the above analysis, a two-factor, nine-item instrument with good psychometric properties for measuring 
the media literacy of elementary school students was developed. This study presented an empirically validated 
model to measure the media literacy of elementary school students. The nine-item MLSS instrument that 
emerged was demonstrated to produce acceptable reliability estimates, and the empirical evidence supported its 
content validity, discriminant validity, and convergent validity. Therefore, this revision of the MLSS instrument 
can be utilised to assess the media literacy of elementary school students. 

The results of this study revealed that female students tend to be more media literate than their male counterparts 
for the 5th and 6th grades elementary students. This result is inconsistent with previous studies (Chang, 2008; 
Lee & Yuan, 2010; Liu & Lin, 2010; Tsai, Lin, & Tsai, 2001; Yen & Lee, 2011) that examined gender differences 
in technology-related attitudes, which have generally indicated that male students held more positive attitudes 
towards technology than did female students. There are several possible explanations for the superior media 
literacy of the female students. First, boys may spend more time outdoors with their peers, playing sports and 
hanging out (Lemish, Liebes, & Seidman, 2001), whereas females spend more time reading, writing and 
listening to music (Trainor, Delfabbro, Anderson, & Winefield, 2010). Second, girls may use media in more 
diverse ways, spreading their literate activities over the different modalities (Unlusoy, de Flaan, Leseman, & van 
Kruistum, 2010), whereas boys may still be more focused on related new technology such as mobile devices 
(Yen & Lee, 2011). Finally, male students may view technology as a playful toy, whereas female students may 
treat it as a tool to accomplish a task (Lee & Yuan, 2010). Therefore, girls may demonstrate a higher level of 
engagement in media literacy, allowing them to outscore boys on both subscales of the MLSS. The media 
literacy practices of girls, who may seem to be in a disadvantaged position in relation to new media, are put in a 
different light when their use of new media is placed within the broader spectrum of media use. Consequently, 
educators should take gender differences into consideration when developing media instructional activities. For 
example, robotics may be integrated into learning activities to promote male engagement in media literacy (Liu, 
2010 ). 

With regard to grade level, both fifth and sixth graders acquired similar levels of knowledge and skills and had 
similar attitudes and perceptions towards media literacy. Accordingly, grade (or age) differences were not 
determining factors of this ability. In general, higher grade students are more media literate than lower grade 
students. However, it seems not so trivial or irreverent in our case. One possible explanation for this result is that 
both fifth and sixth graders had similar experiences about using media and they were just one grade difference. 
For example, they have to understand the basic operations of word processing software in their computer classes. 
Future studies may compare students with a larger grade difference to see if the result is still the same. 

This study presents a convenient tool to assess the perceptions towards media literacy of elementary school 
students, based on learning with media and media communication & ethics. Using this tool, teacher educators 
and researchers can more deeply explore the role that views about media literacy plays for elementary school 
students. Even though the rigorous validation procedure allows us to develop a general instrument for measuring 
media literacy, this work has some limitations that could be addressed in the future. First, while the valid 
instrument was developed using sample data gathered in Taiwan, a cross-culture validation (using another large 
sample gathered elsewhere) would be required for further generalisation of the instrument. Additionally, the 
sampling method has potential bias, as a sample of willing respondents may not generalise to the population of 
all students. Consequently, other samples from different areas or nations should be gathered to confirm and 
refine the factor structure of the MLSS instrument and to assess its reliability and validity. 
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Appendix A 

I tems of the MLSS _ 

Coding_Item_ 

m-l-a I can understand different types of media (e.g. visual media, audio media) and their principles. 

m-1 -b I can understand how to operate media. 

m- 1-c I can understand the content that media convey. 

2 a I can be familiar with the operational functions of media equipment to broadcast the learning 
content. 

m-2-b I can use different media technologies to store/backup the content. 

m-2-c I can discuss with others the content that media display. 

m-2-d I can select appropriate media to edit the messages that I want to convey. 
m-3-a I can use media to carry out daily learning. 

m-3-b I can use media appropriately to convey ideas (e.g. use a camera to record events). 

m_3_c I discuss the displayed contents of media with others. 

m-4-a I possess the accurate understanding of media use. 

m-4-b I can cherish and conserve media equipment. 

m-4-c I can comply with the intellectual property rights of media use. 
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